Skip to content

test: upgrade e2e test framework#985

Merged
wenytang-ms merged 23 commits intomainfrom
wenyt/fixui
Apr 1, 2026
Merged

test: upgrade e2e test framework#985
wenytang-ms merged 23 commits intomainfrom
wenyt/fixui

Conversation

@wenytang-ms
Copy link
Copy Markdown
Contributor

No description provided.

wenytang-ms and others added 22 commits March 30, 2026 14:55
- Rename/delete commands have 'when: false' in command palette, so
  executeCommand cannot find them. Use right-click context menu instead,
  matching the approach used in the old UI test suite.
- Skip 'create with no build tools' test because scaffoldSimpleProject()
  opens a native OS file dialog that Playwright cannot automate.
- Add selectContextMenuItem() helper to VscodeOperator.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The .action-item div is a presentation container; VS Code only handles
click events on the inner <a role='menuitem'> anchor. Use getByRole to
find and click the correct element, and wait for menu dismissal to
confirm the click registered.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Playwright's click() does not reliably trigger actions on VS Code
Electron context menu items (the menu stays open). Switch to:
1. Right-click to select tree item and set focusedView
2. Escape to close context menu (returns focus to tree)
3. F2/Delete keyboard shortcut to trigger rename/delete commands

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
VS Code menus internally handle mouseup events, not click events.
Playwright's simulated .click() in Electron does not reliably trigger
the mouseup handler. Use dispatchEvent('mouseup') to directly dispatch
the event that VS Code's menu system listens for.

Previous approaches that failed:
- .click() on .action-item container: menu stays open
- getByRole('menuitem').click(): menu stays open
- keyboard shortcut (F2/Delete): focusedView not set after Escape

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Neither .click(), dispatchEvent('mouseup'), nor keyboard shortcuts
reliably trigger VS Code Electron context menu items via Playwright.

Switch to raw page.mouse.click() at the element's bounding box center
coordinates, which sends CDP-level InputDispatchMouseEvent that
Electron's native event handling processes correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…dialogs

Root causes found and fixed:
1. Context menu click requires hover first to trigger VS Code's menu
   focus state (.action-item.focused), then click works reliably.
2. redhat.java shows an Electron native dialog (dialog.showMessageBox)
   for refactoring confirmation on file rename. Playwright Page API
   cannot interact with native dialogs. Monkey-patch showMessageBox
   in the Electron main process to auto-return OK.

Verified locally: both rename and delete tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux the button order in dialog.showMessageBox differs from Windows,
so response:0 selects 'Show Preview' instead of 'OK'. Fix by scanning
the buttons array for the 'OK' label.

Also handle the Refactor Preview panel (Apply/Discard) as a fallback
in case the dialog still enters preview mode.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On Linux CI (headless, no recycle bin), the delete confirmation uses
'Delete' or 'Move to Trash' as button labels, not 'OK'. Expand the
regex to match all known confirm labels.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wenytang-ms wenytang-ms marked this pull request as ready for review March 31, 2026 13:02
@wenytang-ms wenytang-ms requested a review from Copilot April 1, 2026 01:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the extension’s UI/E2E testing from vscode-extension-tester to a Playwright-based Electron harness, adding new E2E operators and test suites and updating CI workflows to run them.

Changes:

  • Add a Playwright Electron fixture + operators (generic VS Code + Java-specific) and initial E2E test suites.
  • Introduce a Playwright config + global setup to provision VS Code and install required extensions/VSIX.
  • Wire E2E execution into package.json and switch Linux/Windows CI workflows to run the new Playwright E2E job and upload test artifacts.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
tsconfig.json Excludes the new test/e2e tree from the root TypeScript build.
test/simple/.project Updates Eclipse project metadata (formatting + filtered resources).
test/e2e/utils/vscodeOperator.ts Adds generic VS Code UI helper operations for Playwright.
test/e2e/utils/javaOperator.ts Adds Java-specific UI helpers (LS readiness, tree expansion, etc.).
test/e2e/utils/constants.ts Centralizes E2E timeouts/roles/constants.
test/e2e/tests/projectExplorer.test.ts Adds E2E coverage for Java Projects explorer behaviors.
test/e2e/tests/libraries.test.ts Adds (currently skipped) E2E scaffolding for library/project creation flows.
test/e2e/tests/fileOperations.test.ts Adds E2E tests for create/rename/delete operations in Java Projects view.
test/e2e/playwright.config.ts Defines Playwright runner settings, reporting, artifacts, and global setup.
test/e2e/globalSetup.ts Downloads VS Code and installs redhat.java (+ optional VSIX) pre-test.
test/e2e/fixtures/baseTest.ts Implements the Playwright fixture that launches VS Code via Electron and manages per-test workspace setup/teardown.
package.json Adds test-e2e script and introduces @playwright/test dev dependency.
.github/workflows/windowsUI.yml Switches Windows CI job from old UI tests to Playwright E2E + artifact upload.
.github/workflows/linuxUI.yml Switches Linux CI job from old UI tests to Playwright E2E + artifact upload.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Remove unused imports in fileOperations.test.ts and libraries.test.ts
- Clean up skipped test bodies and dead beforeEach in libraries.test.ts
- Fix JSDoc comment in vscodeOperator.ts to reflect actual CSS fallbacks
- Increase Playwright timeout from 180s to 240s for LS readiness margin
- Remove cliArgs spread from Electron launch (avoid CLI-only flags)
- Remove VSIX install from globalSetup (extensionDevelopmentPath suffices)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@wenytang-ms wenytang-ms merged commit c942b47 into main Apr 1, 2026
6 checks passed
@wenytang-ms wenytang-ms deleted the wenyt/fixui branch April 1, 2026 04:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants